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The primary ourpose of this study was to obtain estimates of the human attribute re- 
quirements of the job elements of Form B of the Position Analysis Questionnaire (PAQ) » 
A secondary purpose was to explore the reliability of job-related ratings as a func- 
tion of the number of raters. A taxonomy of 76 human attributes was used, and 
ratings of the relevance of these attributes to each cf the PAQ job elements were 
obtained. A minimum of 8 raters and a maximum of 11 raters rated each of the attri- 
butes. The mean and median ratings for each attribute as it related to each job 
element were computed, these serving as the basis for the attribute profiles of the 
job elements. Measures of the reliability of the ratings were also gained. The 
conclusions drawn from these analyses were: (1) Raters are able to 'Jte the relevance 
of human attributes to the Job elements in a structured job analysis instrument such 
as the PAQ, and do so with respectable reliability; and (2) a large number of hunan 
attributes appear to be relevant to one or more of the Job elements of the PAQ, with 
some attributes being relevant to more job elements than others. The Implications 
drawn from these conclusions were; (1) the attribute profiles of the job elements of 
the PAQ would seem to be of potential use as the basis for determining the attribute 
requirements of jobs, through job analysis with the PAQ; and (2) in turn, such pro- 
files might be of use in the establishment of the synthetic validity of tests for use 
in personnel programs. The determination of the reliability of the ratings as a 
function of the number of raters was accomplished through the use of the Spearman- 
Brown formula, and this analysis resulted in the conclusion that 8 to 10 raters 
usually would provide ratings of reasonably adequate reliability. 
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ABSTRACT 

The primary purpose of this study was to obtain estimates of the ^ 
human attribute requirements of the job elements of Form B of the Position 
Analysis Questionnaire (PAQ), a structured job analysis instrument based 
on a worker oriented approach. A secondary purpose was to explore the 
reliability of job related rating? as a function of the number of raters. 

This study is part of a larger effort aimed at refining and updating 
a body of data relating to the PAO, and was designed to complete the 
data base necessary for the development of attribute profiles for the 
job elements of the PAQ. (The present study was, in effect, an extension 
of a similar study with an earlier form of the PAQ, Form A.) A taxonomy 
of 76 human attributes was used, and ratings of the relevance of each of 
these attributes to each of the PAO job elements were obtained. A minimum 
of 8 raters and a maximum of 11 raters rated each of the attributes* The 
mean and median ratings for each attribute as it related to each job 
element were computed, these serving as the basis for the attribute pro- 
files of the job elements. Measures of the reliability of the ratings 
were also obtained. The conclusions drawn from these analyses were: 

(1) Raters are able to rate the relevance of human attributes to 
the job elements in a structured job analysis instrument 
such as the PAQ, and do so with respectable reliability (the 
reliability coefficients of the pooled ratings generally 
ranged front .80 to the upper .90*s). 

(2) A large number of human attributes appear to be relevant to 
one or .more of the job elements of the PAO, with some attri- 
butes being relevant to more job elements than others. 

The Implications drawn from these conclusions were: 

(1) The attribute profiles of the job elements of the PAO would 
seem to be of potential use as the basis for determining the 
attribute requirements of jobs, through job analysis of 1obs 
with the PAQ. 

(2) in turn, such profiles might be of use in the establishment 

of the synthetic validity of tests for use In personnel programs 
in industry. . 

The determination of the reliability of the ratings as a functioi^ I 
of the number of raters was accomplished by stepping-up the intra-class 1 
correlation coefficient for each attribute with the Spearman-Brown pre- * 
diction formula to n's ranging from 2 vaters through 20 raters, and the 
empirical checking of these step-ups by drawing subsamples of different 
n 9 s and computing their reliability* The conclusion drawn from this 
analysis was that 8 to 10 raters usually would provide ratings of reason- 
ably adequate reliability. Such a sample size was found to generally 
produce reliabilities in the .80 f s and .90 9 s. 
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INTRODUCTION 



One of the basic objectives of personnel psychology has been that 
of the selection and placement of Individuals In jobs which they are capable 
of performing adequately* In order to accomplish this, psychologists 
have developed numerous teats which presumably measure a variety of human 
abilities, and have validated these tests against various job-related 
criteria* In carrying out these validations, the major thrust has been 
toward the establishment of empirical criterion-related validity of one 
or more tests for any given Job* This has most commonly involved concurrent 
Validity (using the present employee method of test validation) f but 
sometimes has involved predictive validity (using the follow-up method 
of test validation)* Both of these methods* however, are predicated 
on the availability of large numbers of Individuals in any given job, 
and are faced with such problems as the attrition of employees or of a 
restriction of the range of scores within the sample, not to mention 
the problem of the cost of carrying out such validation* In addition, 
such traditional empirical valuation approaches may be seen as lacking 
a degree of parsimony in the practical sense, as they generally must 
be carried out for each individual job In each specific situation* 

In so far as one can hypothesise some underlying order to the world of 
human work, however, it would seem that those jobs which Involve the same 
basic job-related behaviors should also require the same human attributes 
for successful performance* Thus, it seems reasonable to believe that 
in the identification of valid predictors for specific jobs one could 
move from the conventional dead-center approach of empirical validation 
In each job situation toward an approach based on validity generalization 
or synthetic validation (Lawshe, 1952; Balma, 1959; Ghlselli, 1959; 
McCormick, 1959)* Such a shift should make possible a more parsimonious 
basis for the establishment of valid predictors, and also permit the 
establishment of valid predictors in cases where it has not previously 
been possible or practical to do so* This shift would also be In the 
direction of identifying those human attributes that predispose individuals 
to perform effectively on various kinds of job activities (Mecham and 
McCormick, 1969). 

A basic prerequisite of such a possible approach would consist of 
some systematic procedure for Identifying and possibly quantifying job 
characteristics that might serve as c o i on denominators for determining 
the similarities, and differences, between and among jobs. This Is 
essentially a job analysis process* In this regard, a review of the litera- 
ture relating to job analysis points out a few attempts to develop a 
"trait" type of approach to the analysis of jobs, or, In other words, 
attempts to characterise jobs In terms of the traits or human qualities 
required for successful job performance* One of the earliest such attempts 
was made by VI teles (1922, 1932) In his development of the Job Psychograph, 
Following from this is th# work of the United States Training and Employ- 
ment Service in the development of the Worker Characteristics Form (Stead 
and Shartle, 1940). In addition, che work of Jaspen (1949), Trattner, 
Fine, and Kubls (1955), and McCormick, Finn, and Scheips (1957) have added 
knowledge to this area* 
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Other systematic Job analysis approaches have been directed more toward 
the analyses of Jobs in terms of Job activities and job-related behaviors. 
The J-coeffici^nt, in part f is an example of such an approach (Primoff, 
1955; Wherry, 1955). 

Of central interest to this study is the work in this area that has 
been carried out by McCormick and his students in their development of 
a worker oriented approach to the analysis of Jobs, reflected in the 
development, in sequence, of the Check List of Worker Activities, the 
Worker Activity Profile, and more recently the Position Analysis Question- 
naire (PAQ) . For a detailed description of this line of work, and of 
these various instruments, cee Gordon (1963); Mecham (1968); and McCormick, 
Jeanneret, and Mecham (1969). 

The present investigation is one phase of a research program that 
is, In part, directed toward the refinement of a procedure for establishing 
Job requirements on the basis of synthetic or generalized validity. In 
this research one of the general approaches which has been tried is that 
in which experimental sets of job requirements for any given Job are "built- 
up" for the Job on the basis of data on the estimates of the attribute 
requirements of the individual Job elements of a structured job analysis 
instrument, the Position Analysis Questionnaire IPAQ). The basic steps 
followed in such a procedure are (Mecham and McCormick, 1969): 

1. The Identification of the human "attributes" that are most 
relevant for personnel selection. 
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2„ The rating, for each Job element of the PAQ, of the degree 
to which each attribute is relevant to each job element. 

3. The computation of an index of central tendency (i.e., the 
mean or median) of several ratings of each attribute (as to 
its relevance to each of the Job elements of the PAQ) , 
and the subsequent derivation of an attribute profile 
for each Job element (this consisting of the mean or median 
attribute ratings for all attributes). 

4. The analysis of a given job with the PAQ. 

5. The computation, for each Job, of a composite Job attribute 
profile based on some summation or f 1>uilding-up" of the attri- 
bute profiles of the job elements that are part of the job. 

Purpose and Scope of the Present Study 

The present study was primarily directed at obtaining estimates of 
the human attribute requirements for each of the various job elements 
of Form B of the PAQ. Similar data have been collected with respect to 
Form A of the PAQ (Mecham and McCormick, 1969) , but the changes In the 
instrument which resulted In Form B render the earlier data incomplete 
with respect to Form B, thus necessitating the obtaining of data on the 
new Job elements in order to complete the data set. In addition, an 
inspection of the taxonomy of attributes used previously , when compared 
to recent work In the area of the development of taxonomies of human 
^ abilities, pointed out that the previous list should be enlarged for the 

present study. 



The data obtained were also viewed in terms of their possible utilitv 
in exploring the reliability of the ratings obtained as a function of the 
rubber of raters. There is evidence in the literature that the number 
of raters needed for a Riven purpose is dependent upon the sit tint ion, 
and upon the raters themselves (e.g., Christal, Madden, and Hardinp, 
1<>60: Chiselli, 1964), and, therefore, It was thought that the rntiw 
data obtained in this study might be useful in providing additional empi- 
rical data relating to the pooled reliability of ratinps of various numbers 
of raters. 

PPOCEDUFE, 

The determination of what data were relevant for collection, and 
the choice of the method of collecting these data, were in part implied 
by the previous work of Mecham (1968) and Mecham and McCormick (1969), 
and by the differences between Form A and Form B of the PAO. A short 
explanation of these considerations will, therefore, be given at this 
noint . 

The critical difference between the two forms of the PAO, insofar 
as this study was concerned, is that Form B contains 13 job elements 
which were not present in Form A, and an additional 9 job elements which 
were substantially modified from Form A. Minor modifications of job 
elements were not considered to be important enough to warrant further 
Investigation. In order to obtain a complete set of ratings for all 
the job elements of Form B of the PAO for t*< 68 attributes used previously 
by Mecham and McCdrmick, one of two approaches would be necessary: 
(1) obtain new ratings for all of the 68 attributes across all of the 
job elements of Form B of the PAQ; or, (2) use the earlier data and add 
to it ratings of the 68 attributes for the 22 new (or revised) job 
elements. The latter approach was followed in this study due to the 
practical consideration of obtaining the larger number of raters that 
would have been necessary to obtain completely new ratings for all of 
the attributes across all of the job elements of Form B of the PAO. 
The details of this methodology may be found in Marquardt (1972), and 
so will not be repeated here. 

Selection of New Attributes 

It was considered desirable that a list of human attributes for 
experimental purposes such as those in this study should be as comprehensive 
as possible. For this reason, the existing list of 68 attributes was 
reviewed in an attempt to discover any gaps that might exist. The most 
comprehensive recent research relating to the nature of human abilities 
is that carried out by Fleishman and his associates at the American 
Institutes for Research (Theologus et. al. , 1970). The resulting 
taxonomy consists of 37 hu.«an abilities. An examination of this list 
pointed out that. 8 of the abilities were not among those on the existing 
list of 68 attributes. These 8 attributes were included in the pTcsent 
study, and ratings of their relevance to the job elements of the PAO were 
obtained, in addition to obtaining ratings of the 68 "old" attributes 
for the 22 new job elements. (See Appendix A for a list of the 68 "old" 
attributes, and Appendix B for a list of the 8 "new" attributes.) 



Selection of Raters 



The raters in this study were largely drawn from among members of the 
Division of Industrial and Organizational Psychology (Division 14) of the 
American Psychological Association, who were solicited for their cooperation 
in this endeavor. The two considerations which guided the choice of Division 
14 members as the population from which to draw the raters for this study were 
as follows: (1) this is the same population as that from which Mecham drew 
his raters 9 and the combination of ratings of the new job elements with Mecham' s 
ratings would probably be more valid if this were also the population from 
which the raters for this study were drawn; and 9 (2) the rating task which 
the raters were asked to perform was a relatively difficult one, and it was 
felt that this population of raters would most easily and accurately understand 
what was desired of them, and make their ratings accordingly. Materials were 
sent to a total of 97 raters, and returned by 80 raters. 

Rating Procedures 

Each rater was sent a package of materials which Included a set of 
instructions for the rating task, Attribute Rating Forms on which to record 
his ratings, and a copy of Form B of the PAQ. The raters were divided into 
two groups. One group of 45 rated the "old" attributes (those Included In 
the original list of 68) as they related to the 22 new job elements of Form B 
of the PAQ; each such rater rated 12 attributes as related to those job ele- 
ments. The remaining 35 raters were asked to rsU two of the "new" attributes 
as they related to the job elements of the PAQ that were used in this process. 
Twelve of the 194 Job elements were not used because they were of a "write-in" 
nature, or otherwise did not lend themselves to such rating. 

In general, the rater was asked to rate the relevance (or importance) of 
each of the assigned attributes to each job element using a 6-point scale 
which ranged from no relevance to extreme relevance . 

Statistical Procedures 

In order to derive pooled estimates of the judged attribute requirements 
of the job elements of the PAQ, the mean attribute ratings across raters 
were determined for each PAQ job element and attribute. An inspection of 
the distributions of the ratings showed that a few were badly skewed, and 
suggested that in those cases the mean ratings might not be as appropriate 



T*he following format was suggested as an aid for use In making the 
ratings: "(The given attribute) Is of (degree) relevance in, or in dealing 
with, (the specific PAQ job element) ." In addition, the frame of reference, 
or conceptual format, suggested was for the rater to consider a hypothetical 
job in which the highest level of the PAQ job element applied, and then rate 
the attribute as it would be relevant to that aspect of the job represented 
by the particular PAQ Job element. "Highest level" in this context refers 
to the scales provided in the PAQ which are used when analysing a Job with 
the PAQ, and, in particular, to the highest value or top of each of these 
scales* For a more detailed explanation of these procedures, see Marquardt 
(1972) • 
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a measure of central tendency as the median. Although such skewed distri 
L^were rare in the data, it was felt that in these few cases the 
neSian miSt II I more appropriate index to use, considering the future 
It of the data derived from this study. In so far as the means derived 

hute using the following decision rules: 



1. 



2. 



When an even number of rank-ordered ratings were considered, 
and when the two ratings in the middle of the sequence were dif- 
ferent, the midpoint between the two was taken as the median. 

When an odd number of ratings were considered the middle rating 
of the rank-order sequence was taken as the median. 



3 When the median exceeded the mean by .50 or more, «« ff ij len ; 
' variability was assumed to be present to warrant the adjusting 
of the median downward by .SO. 

A When the median and mean were equal, within .49 of each other, 
or when the mean exceeded the median, the median value as com- 
puted was accepted. 

in those cases where the distributions of ratings were symmetrical this 

-i „«e«»nMflllv resulted in equal means and medians, and when the 
S2mm£T£? Positive" skeweu th. procedure resulted in the acceptance 
if II. median m computed. when the distributions were negatively skewed, 
ho^verTtne Jdian would exceed the mean when only decision rules land 2 

...Iij.^.h Decision rule 3 was included to make the final median 
ore Ti e lower) when the distributions of the rating were 

SHtSSTSIS. The'ratlonale for this procedure was a R ain a consideration 

despite the few skewed distributions.) 

In order to determine the stability of the attribute ratings, the 
er reliability for each attribute was computed using the AMOVA 
inter-rater rei "°""J T .;;;* ,,g 71 Sec 4.5). These computations were 

iE : SH SsTs as. ass 

Tne efficient computed was the intra-class correl ation coef fie en , and he 

iHuch frames of reference should not be considered a part of the error 
5 measureTnt when computing a reliability coefficient. O t ^ be 
noted here that all of the reliability computations in this study were ma< 
Tilt the unbiased estimates rather than the biased Anally .the 

"pea^an-Brown prediction formula estimates of the reliability coefficient 
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for the n raters for a given attribute were computed using the Intra- 
class correlation coefficient as a starting point for the step-up. 



The reliability analyses conducted in this study were made on what 
may be termed the "combined" data, that is, the combination of Mecham 
and McCormick s data with the new data collected in this study. Since 
the data collected for the 68 "old" attributes was for only the 22 new 
job elements, and because these data essentially fit the "holes" in Mecham 
and McCormick's data, each of their raters for a given attribute was 
paired with one of the new raters, and the data combined by "filling 
the holes. ' This procedure resulted in a set of ratings for a given rater 
which are complete with respect to the job elements of Form B of the PAO. 

The last analyses performed dealt with the pooled reJ . >f 
varying numbers of raters in providing estimates of job-re data 
such as those obtained in this study. These analyses consisted of: 
(1) drawing a subsample of a few attributes; (2) for each of these attri- 
butes drawing successive multiple subsamples of a given n (for example, 
multiple samples of 2 raters, 3 raters, 4 raters, etc.); and (3) computing 
the reliability of each of these subsamples. The resulting multiple 
reliability estimates for any given n were then averaged and us*d in 
conjunction with the Spearman-Brown formula to estimate the reliability 
at a higher n. This was essentially a comparison of the empirically derived 
values with estimates based on the Spearman-Brown formula to determine 
the degree of correspondence. After this, the Spearman-Brown step-u P s 
of the intrp-class correlation coefficients derived from the total sample 
of raters for each of the attributes were determined for values of n 
ranging from 2 raters through 20 raters. These data were then plotted 2 . 

RESULTS 

As explained above, the median attribute rating for each attribute 
and job element was computed as an index of the attribute requirement of 
the Job element. The complete matrix of these medians may be found in 
Appen'Hx C. The table of means may be found in Marquardt (1972). As 
indi ... d before, however, the Pearson product-moment correlation between 
the x, as and medians across all attributes and job elements was computed, 
and found to be .97. Table 1 reflects the nature of these data, and contains 



Although the determination of the sample size needed to obtain ade- 
quate reliability was made in this fashion, the possibility of making 
this determination by means of the power function charts for ANOVA main 
effects was also considered. Intuitively, this determination could be 
made using power functions, as the ANOVA approach to determining relia- 
bility essentially results in a test of the main effect of differences 
between ratings, and the amount of variance accounted for by this effect. 
However, in this situation the number of treatment levels (items) is 
180, and the highest value of k (number of treatment levels) available frc 
the power charts is k-8. Thus, the power charts are of little value in 
this particular situation, and the approach described was used. This 
approach is, however, the conventional way in which the number of raters 
needed to obtain a certain reliability of the ratings is determined. 
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a sample of the means and medians of three attributes for ten job elements. 
These values may be interpreted using the following 6-point scale, 
which was also the scale on which the raters made their ratings: 

Code Degree of relevance of attribute to job element of PAO 

0 Attribute is of no relevance to job element 

1 Very limited or nominal relevance x 
,inr ted relevance 

if -rate relevance 
& Substantial or considerable relevance 
5 Extreme or extensive relevance 



Table 1 

Mean and Median Ratings of Attribute Requirements 
for Three Illustrative Attributes for a Sample of PAQ Job Elements 



Attribute 



(1) Verbal (42) Variety (69) Ideational 

comprehension of duties fluency 



PAO elements Mean Median Mean Median Mean Median 



1. 


Written materials 


4.7 


5.0 


3.1 


3.0 


0.8 


0.0 


36. 


Decision making 


4.5 


5.0 


3.8 


4.0 


3.2 


3.5 


58. 


Measuring devices 


1.2 


1.0 


1.8 


2.0 


0.4 


0.0 


81. 


Assembling/disassembling 


0.5 


0.0 


1.9 


2.0 


0.9 


0.5 


93. 


Finger manipulation 


0.5 


0.0 


2.0 


2.0 


0.7 


0.0 


108. 


Signaling 


2.5 


3.0 


1.8 


2.0 


1.2 


0.5 


133. 


Staff functions 


4.4 


4.0 


4.0 


4.0 


3.9 


4.0 


148. 


Civic obligations 


2.9 


3.0 


2.5 


2.5 


2.8 


3.0 


170. 


Repetitive activities 


0.6 


0.0 


1.9 


1.5 


0.4 


0.0 


174. 


Precision 


1.4 


1.0 


2.1 


1.5 


0.7 


0.0 



Table 2 reflects the number of raters who rated each given attribute, 
and the reliability coefficient for each of these attributes. These 
coefficients were computed across 180 of the 194 job elements of Form B 
of the PAQ, because job elements 188 through 194 deal with pay/income, 
and no d&ua were collected for them; and job elements 44, 60, 127, 160, 
and 181 were eliminated from consideration as they are open ended in nature 
and any sort of response is possible. (Job elements 143 and 176 were also 
omitted as data for these two job elements were collected separately.) 
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Table 2 



Coefficients of Reliability of Ratings of 
Attribute Requirements of 180 Job Elements of the PAQ 



Attribute 
number 



Attribute 



1. Verbal comprehension 

2. Word fluency 

3. Oral communication 

4. Numerical computation 

5. Arithmetic reasoning 

6. Convergent thinking 

7. Divergent thinking 

8. Intelligence 

9 . Long-term memory 

10. Short-term memory 

11. Aesthetic Judgment 

12. Visual form perception 

13. Perceptual speed 

14. Closure 

15. Movement detection 

16. Spatial visualization 

17. Near visual acuity 

18. Far visual acuity 

19. Depth perception 

20. Color discrimination 

21. Auditory acuity 

22. Olfactory acuity 

23. Gustatory acuity 

24. Tactual acuity 

25. Body orientation 

26. Kinesthesis 

27. Finger dexterity 

28. Manual dexterity 

29. Arm/hand positioning 

30. Arm/hand steadiness 

31. Continuous muscular control 

32. Rate of am movement 

33. Eye-hand coordination 

34. Eye-hand-foot coordination 

35. Simple reaction time 

36. Response integration 

37. Dynamic strength 

38. Static strength 

39. Explosive strength 

40. Rate control 



No. of Intraclass 
ra ters correl at ion 
("N")* coefficient 



Coefficient 
for 

("N") raters** 



11 


. 709 


.964 


Q 


.635 


.940 


10 


.705 


.960 


10 


524 


.917 


Q 


511 


.904 


10 


.409 


.874 


10 


497 


. 908 


10 


618 


.942 


10 


468 


.898 


1 0 


370 


.855 


0 


. J J J 


.818 


Q 


AAQ 




1U 


A1 1 
• *f 1 j 


875 


i n 
lu 




869 


in 
1U 


A 9 A 


881 


i n 
lu 


A7S 


.900 




11 7 
• 31 / 


91 L 
• Jin 


1U 


^97 


.918 


i n 




.929 


in 

1U 


A71 


.899 


Q 


5 AO 


.926 


ii 


.520 


.923 


ii 


.378 


.870 


10 


.602 


.938 


10 


.361 


.849 


10 


.694 


.958 


10 


.571 


.930 


10 


.593 


.936 


10 


.647 


.948 


10 


.552 


.925 


10 


.563 


.928 


9 


.603 


.932 


11 


.596 


.942 


11 


.497 


.916 


9 


.303 


.796 


9 


.521 


.907 


10 


.609 


.940 


11 


.628 


.949 


10 


.358 


.848 


10 


.397 


.868 



*This number refers to the number of raters in the combined data 
**Computed from the intra-class correlation coefficient with the 
Spearman-Brown prediction formula 
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Table 2 (continued) 



Attribute 
number 



Attribute 



No* of 
raters 



Intraclass 

correlation 

coefficient 



Coefficient 
for 

("N") raters** 



41. Mechanical ability 

42. Variety of duties 

4 3 . Repet i t ive /sh ort -eye le 
operations 

44. Dealing with things/objects 

45 . Processes /machines/ techniques 

46. Scientific/technical activities 

47. Dealing with people 

48. Social welfare - 

49. Influencing people 

50. Directing'/ controlling/planning 

51 . Empathy 

52. Personal risk 

53. Conflicting/ambiguous 

information 

54. Pressure of time 

55. Sensory alertness 

56* Attainment of set standards 

57. Working under specific 

instructions 

58. Working alone 

59. Separation from family/home 

60. Stage presence 

61- Prestige/esteem from others 

62. Tangible/physical end-products 

63. Sensory /Judgmental criteria 

64. Measurable/verifiable criteria 

65. Interpretation from personal 

viewpoint 

66. Susceptibility to fatigue 

67. Dealing with concepts/ 

information 

68. Creative activities 

69. Ideational fluency 

70. Originality 

71. Problem sensitivity 

72. Spatial orientation 

73. Selective attention 

74. Time sharing 

75. Stamina 

76. Speed of limb movement 



10 


.338 


.836 


8 


.211 


.682 


ft 

O 


ArtA 

• tut 


844 


ft 

0 


A1A 


860 


ft 

O 


A in 


.858 


ft 

O 


1AA 


809 


Q 

? 




.960 


ft 

0 


AHQ 


847 


8 


.695 


.948 


8 


.476 


.879 


ft 

O 


AAA 
. ODD 


941 


8 


.528 


.900 


o 

O 


*n 
• jJ/ 


on** 


ft 
o 




767 


ft 
o 




76Q 


ft 
o 


A* A 


»A7 
. oo / 


o 

o 




• /oo 


9 


• UUj 


nAl 


ci 
o 


• ZOO 


7AA 


ft 

o 


. J JO 


Qn? 


ft 

o 


. J 1j 


894 


9 


.331 


.816 


8 


.412 


.849 


8 


.204 


.672 


8 


.359 


.817 


8 


.295 


.770 


9 


.587 


.928 


8 


.229 


.703 


10 


.487 


.905 


9 


.414 


.864 


9 


.450 


.880 


8 


.478 


.880 


9 


.540 


.914 


9 


.510 


.904 


8 


.528 


.900 


8 


.699 


.949 



*This number refers to the number of raters in the combined data 
**Cotnputed from the intra-class correlation coefficient with the Spearman-Brown 
prediction formula 
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Each of the intra-class correlation coefficients reported In Table 2 
may be Interpreted as the reliability of a single hypothetical rating, 
computed from data adjusted for the frame of reference of the raters, and 
is approximately equal to the average Intercorrelatlon between ratings 
given by all possible pairs of raters. Since this statistic only reflects 
the reliability of a single rating, and since the reliability coefficients 
of Interest here are those for the means of all raters, the Spearman-Brown 
prediction formula values of the reliability coefficients for the actual 
number of raters for each attribute are also given in Table 2. Each of 
these coefficients may be interpreted as the reliability of the mean 
attribute ratings across all raters for a given attribute. It might be 
mentioned here, however, that the actual correlation coefficient (rather 
than the Spearman-Brown step-ups) was also computed for each attribute for 
the total sample of raters, and it was found that these values were 
actually the same as those given by the Spearman-Brown formula. 

Table 3 reflects the number of attributes which have intra-class 
correlation coefficients falling into given intervals of .05 starting at 
an initial value of .20, and going through a value of .75. From this 
table, it may be seen that the majority of the attributes have coefficients 
greater than .25. Figure 1 shows the graphs of the Spearman-Brown esti- 
mates for Increasing numbers of raters, as computed using intra-class 
correlation coefficients ranging from .20 through .75 in steps of .05 as 
starting values, and numbers of raters ranging from 2 through 20. As 
is evident from Table 3, these values represent the range of those found 
empirically in this study for the combined data. It was decided to use 



Table 3 



Frequency Distribution of the Intra-class Correlation 
Coefficients for the Combined Data 



Value of intra-class 
correlation coefficient 



Number of attributes 
falling in interval 



Less than .19 



1 

3 
5 
5 
8 

10 
8 

14 
9 
6 
4 
3 



.20-.25 
.26-. 30 
.31-. 35 
.36-. 40 
.41-. 45 
.46-. 50 
.51-. 55 
.56-. 60 
.61-. 65 
.66-. 70 
.71-. 75 
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these values when plotting the graphs, rather than those derived empirically* 
as the data and graphs resulting from such values are equivalent to those 
which would be found using the empirical data, and result In the same 
conclusions. It may be seen from Figure 1 that the curves seem to be 
levelling off somewhat in the range of 8 to 10 raters. 

DISCUSSION AND CONCLUSIONS 

As Is evident from the attribute profiles resulting from this study 
(Appendix C), job activities related to the various job elements of the 
PAQ appear to depend upon a wide range of human qualities. The reliability 
coefficients obtained for the ratings upon which the profiles are based 
were rather respectable, generally ranging from .80 to the upper .90's, and 
give an indication of the stability of the means and medians which were 
computed. 

In reviewing the pattern presented by the reliability coefficients, 
it becomes evident that the attributes of an "aptitude" nature were rated 
more reliably than those of an "interest or temperament" nature. Such a 
finding is consistent with that of prior research, and is, In part, explain- 
able by looking more closely at the actual attributes which were used. 
The attributes which were of an aptitude nature were generally those 
which are commonly referred to in a personnel setting as human abilities, 
and, therefore, those with which the raters In this study might be expected 
to be most familiar. The attributes which were of an interest or tempera- 
ment nature, on the other hand, were generally those which presumably are 
relevant more to the personal Interest in, and personal adjustment to, 
the various job activities and situations depicted by the job elements, 
than to the overt behavioral aspects of the elements as such. Thus, 
their relevance to the job elements might actually be of less consequence 
than that of the "aptitudes. " Such less obvious direct relevance of the 
interest and temperament attributes to the Job elements, in turn, may 
have contributed to the somewhat lower reliability of the ratings of them — 
as contrasted to that of the aptitudes. Some written comments received 
from the raters would seem to support this hypothesis, as the raters In 
general seemed to have more trouble rating the temperament attributes 
than thay did the aptitude attributes. 

In so far as the data concerning the step-up In reliability as a 
function of the number of raters is concerned, the data would seem to 
indicate that if future studies are concerned with the reliability of 
ratings of job-related variables, the researchers would usually be wise 
to obtain samples of raters ranging in size from 8 to 10. The desired 
degree of reliability and the use which is to be made of the data must, 
of course, be taken Into account when the number of raters is determined, 
but, in general terms, fewer than 8 raters would most likely result In 
data of questionable reliability, and more than 10 raters would most 
likely be a waste of time and effort when judged in relation to the 
negligible return in increased reliability balanced against the increased 
costs In terms of time and money. This recommendation is, however, 
tempered by a consideration of the population from which the raters 
are to be drawn, the familiarity of raters from this population with 
the concepts being rated, and the number of items to be rated. 
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In general, the following conclusions appear to be warranted on the 
basis of the results of the investigation: 

1. Raters are able to rate the relevance of human attributes to 
the job elements in a structured job analysis instrument such as 
the PAQ, and do so with respectable reliability (the relia- 
bility coefficients of the pooled ratings generally ranged 

from ,80 to the upper .90's). 

2. A large number of human attributes appear to be relevant to 
one or more of the job elements of the PAO, with some attri- 
butes being relevant to more job elements than others. 

In turn f it would seem that these conclusions might suggest the following 
implications: 

1. The attribute profiles of the job elements of the PAO would 
seem to be of potential use as the basis for determining the 
attribute requirements of jobs, through job analysis of jobs 
with the PAQ. 

2. In turn, such profiles might be of use in the establishment 
of the synthetic validity of tests for use in personnel 
programs in industry* 
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APPENDIX A 
LIST OF ATTRIBUTES DEVELOPED BY ME CHAM 



Attributes of an "aptitude" nature 

1. Verbal comprehension ; ability to understand the meaning of words 
and the ideas associated with them. 

2. Word fluency : ability to rapidly produce words associated with 
a given word. 

3. Oral communication : ability to communicate ideas with gestures or 
with spoken or written words. 

4. Numerical computation : ability to manipulate quantative symbols 
rapidly and accurately, as in various arithmetic operations. 

5. Arithmetic reasoning : ability to reason abstractly using 
quantitative concepts and symbols. 

6. Convergent thinking : ability to select from possible alternative 
methods, the method of processing information that leads to the 
potentially best answer or solution to a problem. 

7. Divergent thinking : ability to generate or conceive of new or 
innovative ideas or solutions to a problem. 

8. Intelligence : the level of abstraction or symbolic complexity 
with which one can ultimately deal. 

9. Long-term nemory: ability to learn and store pertinent information 
and selectively to retrieve or recall, much later in time, that 
which is relevant to a specific context. 

10. Short-term memory : ability to learn and store pertinent information 
and selectively to retrieve or recall, within a brief period of 
time, that which is relevant to a specific context. 

11. Aesthetic judgement : ability to make sensitive evaluations of 
artistic quality in one or more of the following: music, style, 
painting, sculpture, photography, architecture, etc. 

12. Visual form perception : ability to perceive pertinent detail or 
configuration in a complex visual stimulus. 

13. Perceptual speed : ability to make rapid discriminations of 
visual detail. 
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14. Closure : ability to perceptually organize a chaotic or disorgani- 
zed field into a single perception* 

15. Movement detection : ability to detect physical movement of 
objects and to judge their direction* 

16. Spatial visualization : ability to manipulate visual images in 
two or three dimensions mentally* 

17. Near visual acuity : ability to perceive detail at normal 
reading distance. 

18. Far visual acuity : ability to perceive detail at distances 
beyond normal reading distance. 

19. Depth perception : ability to estimate depth of distances or 
objects (or to judge their physical relationships in space). 

20. Color discrimination : ability to perceive similiarities or 
differences in colors or in shades of the same color, or to 
identify certain colors. 

21. Auditory acuity : ability to perceive relevant cues by sound. 

22. Olfactory acuity : ability to perceive relevant cues by smell. 

23. Gustatory acuity : ability to perceive relevant cues by taste. 

24. Tactual acuity : ability to perceive relevant cues by touch. 

25. Body orientation : ability to maintain body orientation with 
respect to balance and motion* 

26. Kines thesis : ability to sense position and movement of body 
members* 

27. Finger dexterity : ability to manipulate small objects (with the 
fingers) rapidly and accurately* 

28. Manual dexterity : ability to manipulate things with the hands. 

29. Arm/hand positioning : ability to make precise, accurate movements 
of the hands and arms. 

30. Arm/hand steadiness : ability to keep the hands and arms immobilized 
in a set position with minimal tremor. 

31. Continuous muscular control : ability to exert continuous control 
over external devices through continual use of body limbs* 

32. Rate of arm movement : ability to make gross, rapid arm movements. 
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33. Eye-hand coordination : ability to coordinate hand movements with 
visual stimuli. 

34. Eye-hand-foot coordination : ability to move the hand and foot 
coordinately with each other in accordance with visual stimuli. 

35. Simple reaction time : the period of time elapsing between the 
appearance of any stimulus and the initiation of an appropriate 
response • 

36. Res ponse integration : ability to rapidly perform various appro- 
priate psychomotor responses in proper sequence, 

37. Dynamic strength : ability to make repeated, rapid, flexing 
movements in which the rapid recovery from muscle strain is 
critical* 

38. Static strength ; ability to maintain a high level of muscular 
exertion for some minimum period of time. 

39. Explosive strength : ability to expend a maximum amount of energy 
in one or a series of explosive or ballistic acts (as in throwing, 
pounding, etc.) 

40. Rate control : ability to make continuous anticipatory motor 
adjustments, relative to change in speed and direction of 
continuous moving objects. 

41. Mechanical ability : ability to determine the functional inter- 
relationships of parts within a mechanical system. 

Attributes of an interest or temperament nature, as characterized by 
different types of job situations to which people must adjust . 

42. Variety of duties : duties often, characterized by frequent change. 

43. Repet itlve /shor t-cycle operat ions : operations carried out 
according to set procedures or sequences. 

44. Dealing with things/objects : preference for situations involving 
activities which deal with things and objects rather than 
activities concerned with people or the communication of ideas. 

* 

45. Processes/machines/ techniques : situations which are nonsocial in 
nature, being primarily concerned with methods and procedures 
often of a mechanical or chemical nature. 

46. Scientific/ technical activities : using technical methods or 
investigating natural phenomenon using scientific procedures. 
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47. Dealing with people : i.e., personal contacts beyond giving and 
receiving instructions. 

48. Social welfare ; working with people for their presumed good. 

49. Influencing people ; influencing opinions, attitudes, or judgements 
about ideas or things. 

50. Directing/controlling/planning ; operations involving the 
activities of others, or processes with which others are involved. 

51. Empathy : seeing things from another person *s point of view. 

52. Personal risk ; risk of physical or mental Illness or injury. 

53. Conflicting/ambiguous information ; ability to tolerate and 
critically evaluate information of an u? certain or opposing 
nature. 

54. Pregj-jre of time : working in situations where time is a critical 
factor for successful job performance. 

55. Sensory alertness ; alertness over extended periods of time. 

56. Attainment of set standards : attainment of set limits, tolerances, 
or standards. 

57. Working under specific instructions : i.e., those that allow little 
or no room for independent action or judgement in working out job 
problems. 

58. Working alone : working in physical isolation from others, 
although the activity may be integrated with that of others. 

59. Separation from family/home : separation for extended periods of 
time. 

60. Stage presence : speaking to or performing for an audience. 

61. Prestige/esteem from others : working in situations resulting in 
high regard from others. 

62. Tangible /physical end-products : working with material elements 
or parts which ultimat ^x) result in a physical product . 

63. Sensory /j udgmantal criteria : arriving at generalizations, 
judgments, or decisions which require sensory discrimination or 
cognitive appraisal. 

64. Measurable/verifiable criteria : arriving at generalizations, 
judgments, or decisions based on known or obtainable standards, 
characteristics, or dimensions. 
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65 . I nterpretation from personal viewpoint : interpretation of feelings, 
ideas, or facts in terms of personal viewpoint or values. 

66. fiuflce ptibilitv to fatigue ; u JM inished ability to do work, either 
physical or mental, as a consequence of previous and recent work 
done. 

67> D ealing with concepts/information : preference for situations that 
involve conceptual or informative ideas and the possible communica- 
tion of these ideas to others. 

68 t creative activities ; preference for situations involving the 
finding of new solutions to a problem or new modes of artistic 
expression. 
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APPENDIX B 
LIST OF NEW ATTRIBUTES 



New attributes of an "aptitude 0 nature 

Ideational fluency ; the ability to produce a number of ideas 
concerning a given topic. This attribute is only concerned with 
the number of ideas produced and does not extend to the quality 
of those ideas. 

2. Originality : the ability to produce unusual or clever responses 
related to a given topic or situation. This attribute is concerned 
with the degree of creativity of responses and does not dear' with 
the number of responses made. 

3. Problem sensitivity ; the ability to recognize or identify the 
existence of problems. This attribute does not include any of the 
reasoning necessary for the solution of a problem. 

4. Spatial orientation : the ability to maintain one f s orientation 
with respect to objects in space or to comprehend the position of 
objects in space with respect to the observers position. 

5. Selective attention : the ability to perform a task in the presence 
of distracting stimulation or under monotonous conditions without 
significant loss in efficiency. 

6. T ime sharing : the ability to utilize information obtained by 
shifting between two or more channels of information. The infor- 
mation obtained from these sources is either integrated and used 
as a whole or retained and used separately. 

7. Stamina : this ability involves the capacity to maintain physical 
activity over prolonged periods of time. It is concerned with the 
resistance of the cardio- vascular system to breakdown. 

8. Speed of limb movement : this ability involves the speed with 
which discrete movements of the arms or legs can be made. The 
ability deals with the speed with which the movement can be carried 
out after it has been initiated; it is not concerned with the 
speed of initiation of the movement. 
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APPENDIX C: MEDIAN ATTRIBUTE RATINGS 

FOR JOB ELEMENTS OF THE 
POSITION ANALYSIS OUESTI ONNAI RE (PAO) 



MEDIAN ATTRIBUTE RATINGS FOR JOB 
OF THE POSITION ANALYSIS QUESTIONN 



Attributes 
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Elements 
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. 3 3.3 3. J 3,fc 1.3 > .6 9.9 9 



.3 9.9 1.3 8.3 t.9; ?, 
.4 0.3 4.1 3.3 ,.8 '. . 

.* t.o o.t ?.o .*» 



1.9 9.3 0 



0 9.3 

o o.o 

0 9.9 

,3 9.1 
»fc 9.3 
3 9.9 



— r 

0.9,9- 
1.9)3. 
?.« t. 
I.ljl 
1.9 S 



3.9 1.9 
9 3.9 3. 3 9.9 9 
9 9.9 1.0 9.9 9.3 
9 9.9 9.3 0.9 9.1 

9 3.4 3.0 9.9 3 

P J, 3 3.9 a. 3 1 



- s?t.: 

1.3 1.4 ?. 3 1.9 
1.3 7.9 3.* S.9 
2.3 ?.t 1.8 1.9 
1.9 7.4 2.t 1.9 
7.3 7.3 1*9 1.9 
1.9 7.5 7.9 l.t 
S.S 1.9 ' 7 8 



>.: ?,s t.a t.a « 



9 9.3 3.C 9.9 3.0 
9 1.9 9.9 9.9 9.3 
9 9.1 9.9 9.9 1.9 
9 9.0 9.0 9.1 9.0 
6 9.9 9.9 9.9 0.0 



?.s 

• 7.4 3. 

i 1.9 3. 

■ 7.4 ». 

i 3,4 3, 

!?*4 9. 



3 9.3 
3 9.9 



9. 9 
9 9.9 



?.e t.i.t.o t.s 

S.t J.t t.t 1.9 

2.9 I.* t.l 7.9 

7.9 1.9 9.9 t.t 

3.9 1.* 9.3 



... ?.? 2.3. a.c i.^Lfje 



9 3.9 9.9 0.9 t.Ht.4 3.0 7.0 t.9 

s 9.0 3,, 



*-8 ?•« t.t 



?.0 t 1.9 9-0 9.9 0.9 7,0 



1.9 3.3 9.9 

'.9 t.9 3 0 



9. 9 3.3 9.? 
9.9 3.9 9*0 
9.9 1.3' 9*9 
0.0 t.4j 1.9 
t.9 t.41 t.9 
t.9 1.9> 9.9 
9.1 .fct.t 
.4 !,1 



5.9 t.3 " 
•.9 *.3 
l.t 



»>.9 ?.t 1.9 

7.4 M t.l 

7,4 7.4 l.t 

7.4 7.S 1.9 



4.9 1. 

4.9 J, 

t.9 J- 

t.S t. 

t.S 9, 
7.4 



9 S.9 
9 1.4 
9 7.9 
9 7.0 
3 7.4 
t t 



I 

7.3 *.9j t.4 7.9 4.t 
9.0 t.Sj 1.4 1,4 4.| 

1.4 t.t l.t t.t S.S 
t.t t.9| S.t 9.9 4.1 
9.3 t.t S.4 1,9 4.9 

7.9 1.9 9.9 S.t 

*** 



S.4 3, 
1.9 3, 
S.t I« 
l.t 



J. 9 1. 



9.9 

1.9 7>4 9.9 

4.9 7.4 9.9 

4.9 3.4 7.9 

t.9 7.4 9.3 

t.t ?.« 3.9 

1.4 7,3 1.3 



C7C373 .4 7.3 ».5 f.j e.; 7, s 1.1 .4 j.t ±.3>S.S !,* t.9 i.3 9.} 3. 

3,9 3.9 .4 1.8 »-4 7.4 3.2 7.9 t.9 .4 9.0 C.3S.S t.S *.9 3 3. 

9.9 9.9 1.9 7.9 t.4 7.4 «.t 7.9 7.9 t.t f.9 1.3 J. 4 t.4 t.t S.t 9.9 9. 

9.9 9.9 7.9 J.I 4.9 7.4 3.4*7.9 S.t t.9 .4 O.Ojl.t .4 t.t S.9 9.9,9. 

9.B 9.9 1.5 *.t t,$ 3.4 7.4 7.4 7.t S.t 9.9 9,9 t.9 t.t S.9 S.9 .4,9. 

1.9 S.t 1.4 7.0 t.f 7.4 1.4 t.t t.9 t.t 9.9 S.t S.t 7.4 t.9 S.9 9.t|3. 

9.9 9.3 t.4 S.3 t.4 7.4 t.4 t.4 1.0 7.4 9.9 1.9 [3.4 3.4 S.9 S.9 I. 93. 

3.6 9.9 t.9 7.4 ».* ».t ? -9 3.9 ?.t l.t .4 t.t 3.4 t.t 3.9 S.9 t.t I. 



t.t t.9 7.9 t.9 9.9 4.S t.9 t.4 t.9 3.4 7.4 t.t t.9 3.4 3.9 

^4 7*9 lU 1.9 5.9 *,4 t.9 t.t t.t S.4 3.9 t.t t.t t.9 t.9 

7.9 S.9 ?.t t.t 1.9 S.t 7.9 S.9 t.9 S.t! 3.9 »•* J ** *' • 

3.9 1.4 9.3 t.C 8.3 1.4 1.3 t,4 S.4 7.4,7*4 4.9 t.9 t.9 t.t 

1.4 7.9 t.9 .t I. I. t.3 t.t t.4 3.4 ?.4j?.4 4.9 t.t S.4 t.o 

1.45 S.9 t.t 3.6 3.0 7.3 t.t 3.9 S.9 7.8 7.9 t.t 7.4 7.t ?.t 

t.t 3.9 .4 ■ ■? ?-4 1.1 a .8 S.t 1.4 7.9 t.Q S.4 .4 1 " 



4.9 t.t t.9 4.3 .4! 9< 

t.t .4 t.9 t.9 I.I I 

4. t t.9 t.9 t.9 9.9 
t.t t.t 4.9 4.9 .4 
4.9 t.t t.t t.4 t.9 

5. t t.t t.9 t.O 
ttt 3.1 M 



It [tiluti'4 *?#« -roc****. 

Ji JwUiPr. eo*4ttie* $a*lit» 

>3 £att*itl<if «^*iti:v 

14 ratiwttrc »«r* 

n ritlMtint tlm* 



17 ttttwir* IB pr»M*» iolvlrc 

Ji u«#l jlaiM»ir>f,'#t» , «NU»»»« 

\* faaMftlr-f tn'oreatto* 

40 AB*lf»lPf, tnfor»*ttoO 



,3 7.0 .*.4 9.9 7.9 1.9 .4 9.9 9.9 9.9|t.9 7.9 t.4 t.S 

.3 3.0 9.3 3.9 9.3|t.t 3.9 3.« 9.9 9.917.9 7.9 7.4 9.9 7.9 

.2 9.3 9.9 9.0 t.t!9.C t.t 3.t t.t 0.t!7*t 2.4 1.0 .4 9.0 

,4 d.t t.C 3.9 3.9 t.t «,9 9*9 S.t t.9;?.4 S.t t.t t.t t.t 

7.9 3.9 0.3 9.9 9.1't.t 9.9 t.t 3.t t.t! 3.9 3.4 7.9 .4 t.t 

:.3 3.9 t.i ;.s t.t a. a o.t 9.9, s.t ».t t.t .4 ?.q 



7,t t.t t.t t.3 ?.t 

t.t t.t 7.4 t.9 3.4 

t.4 7.3 S.t t.4 S.t 

3*4 ft ?,t 9.8 3.9 

7.t 7*4 S.t 1.9 3.4 

4.9 9.9 .4 0.9 t.4, 



9.9 t.4 1.9 t 

t.O 3.0 7.« l.t 7.4 

t.t 9.1 S.4 t.t 7.4 

t.t t.t t.t t.t t.t 

t.t t.t 7.t t.9 7.t 

9.0 9.6 ?.t .t .t 



l.t 1.4 I.I .« t.i'f.2 t.3 t.3 ).9 .4]7.9 7.9 t 1.3 

7.4 3.4 t.t .4 O.0!7,9 7.9 l.t 3.9 l.tj?.4 7.t 7.t t.t 9.t 

3.1 .4 9.9 .4 9.9 t.t t.4 3.4 t.9 7.9 3.9 t.4 S.4 7.4 7.9 

7,4 .4 '.9 9.9 0.9 J, 4 7.9 6.9 1.9 1.919.9 t.9 .4 1.9 9.t 

t.t 7.4 t.t .4 O.t S.4 7.4 7.4 1.9 .4't.t 7.4 .4 7.8 7.9 

0 1.4 O.t t.O t.t't.t 9.9 3.9 t.t t.t, 9.9 t.t 3^4 7.4 ?.t 



i.l 



.4 S.t 7.9 1. 



1.4 



7.9 t.t t.t 1.9 O.t 9 
S.t 9.9 7.4 S.t S.t 9 
7,t t.t 7.t 7.4 1.9(9 
7.4 t.t 7.4 S.t t.t 9.9 
t t.t l.t t.l t 



tJ Tr#«i««f I"" 



rjatit ton 



*.« 7.9 S.t ?.« l.OJt.C 1.4 ?.t l.t t.t 
*.9 J.t **t 7.t 7.»|t.9 *.t t.t 1.4 



;t.9 3.0 J.t 3.9 l.t 
.9 S.9 1.4 3.4 t.9 
• •■> !,0 S.3 1.3 1.4 
.4 1.9 t.9 1, 



.9 S.9 1.9 

S.4 7.9 

1.3 3.4 

9 t.O 1.3 1 



t.4 7.9 t.t l.t 9. tit.; '.9 1.3 3.0 3.3 

1.3 S.S t.9 7.9 t.t J. 3 1.0 '.3 l.t .4 

t.9 ?.t 1.4 t.t 3.9 ».: l-» »•« 

1.1 7.9 S.t 3.0 ?.l'l * • *> J.t »•* •* 

8.9 l. t S.t S.I 7.4' t.C 1 9 7.J *»_93i0 

— " • ' t I.OTsT 

4 l.t 1 



%.t t.t 1*1 t.t t.9 

1.9 7.9 S.t t.4 1.4 

7.1 1.4 t.t t.4 7.4 

S.t 7.0 3.4 t.* 7.t 

t.O 1.4 1.4 t.9 fc.t 

1.9 7.0 7. 4 t.t 1.4 



9.3 7.9 t.t S.t S.t 

t.3 7.9 3.C t.9 S.t 

1.3 7.t 3*3 3.4 S.t 

1.9 7.1 ?.t 1.4 1.4 

1.9 T- t • 9 t.9 t.t 
S.t 



S.4 l.t t.t t.t t.t t. 

.4 t.t t.9 t. t 9.9 t< 

9.9 t.t t.9 t.t O.t t. 

1.4 9.9 t.9 9.9 l.t) •< 

1.4 0.0 t.l t.9 9.9 9. 

» .0 t.l t.t t.O t.t 3. 



• 1.4 S.t !.• 



7.9 t.t 7.4 t 
t.9 l.t 1.9 S.t,.. . - -_ _ 
7T4~t.ltfr< 7.1 t.t 



».t t.4 t.t t.t t.t 

^«t t.« t.t t.t 

t.4 7.t t.t t.t t.t 

7.4 7.4 t.t l.t t.9 

t.9 7.4 9.0 l.t «.! 

1. 0 7.4 t.t l.t t 



.1 I.l 0.9 .4!».0 9.9 

9.9 3.1 t.9 3.1:3.3 .4 t.t 7.t S.t 

7.4 t.t l.t 9.9 J.CU.5 3.9 t* 9 3.9 3.4 

t.O 9. 9 9.9 9.0 9. 0 9. 9 9.9 7.t S.t 3.4 



O.t t.t 



T*.» t.t *.l 4.9 t.t]t.* 

1:511:? vmu 1:1 H^!- 1 I't 

t.t s.t ..i 7.4 s.t;*.? i.t j.t s.t »•• ••• 

-.0 1.0 t.O 7.9 t.9 t 5 l.t 7.4 7.4 S.t 7.4 1.4 7.4 t.t t.O 

t:! 1:1 J:5 ;:J ;.!,£.- !:' s.t u% s.t ?*t s.t s.o t.o 

4.1 t.t *.o s.t •,*'*.r ».3 s.t *.« I'l 

•..0 1.0 3.9 t.4 7.4j»,C t. 1 " 3.3 3.9 ?■• t.t t.O ».t 7.4 

-.1 t.O *.0 t.t t.i't.C S.t S.i 7.9 ♦•f|>-t t-t t.t t 

- 9 t.O t.O t.O 1.1 t.C 1.0 J.O '.4 J. j .J U\ * 

t S.t t.6 t.9 I.* t.R 1.9 S.t S.4 1 1.9 t.4 t.t t.9 1.9 

9 t.O t.9 4.6 t.* 4.0 '.s 5.9 7.9 t.|i S.6 t.t t.t t.9 S.9 

».0 <..* ».3 3.*; 3.3 l.t 7.9 3.4 7.0 
1.3 " 



7.9 3. 



7.9 t.t 

!.3 S.t 7.9 3.4 



t.4 1.0 1.0 t.O 
t.O t*3 »>9 4.3 S.9 t 
Jj^jOJ.O 3.9 fc.fl_S't t * 
t.3 1.3 3.4 1.9 1.9 1 
t.O t.O 3.9 t.t t.9 1 

t.o t.o t. 3 s.o s.g t 

%.9 t.t t.t 1.9 t.t t.S 1.4 1.3 1.' 

*^.4 S.O 3.9 7.0 1.3 J.O «. 4 1.9 1.9 1.9 

t.l t.O l.t 7.4' S.O -.1 1-3 S.t S.O 

*"'§ t.O S.t t.t t.l'',.C 4.9 1.4 t.t 3.0 

t.9 3.4 7.0 t.t 7.t,t.«. S.4 t.t S.O 1.0 

3.4 7.5 1.0 7.0 t.t!?.* S.t 3.4 .4 
1.1 1.0 t.l 7.t l.t t.C ■•• 

S.t t.t t.l 7.4. O.t J. 5 7-4 7.4 1.0 0.0 

S.t t.O t.O 7.0 S.lil.C 7.4 7.4 l.t t.l 

*>.t t.9 3.0 t.O 7.9<t.O S.S t.9 1.9 t.t 

7.4 l.t ?.t S,Q j.t, ?,fl l.t 1.6 



t.4 7.0 t.9 9.9 7.4 t.«> 1.9 t.t t.t 7.4 

L, *.4 t.O t.t 7.t S.t *.? 1.4 7.4 7.9 7.9 

9 t.t t.O t.4 7.1 t.t t.« 1.9 S.O l.t 7.0 

t.t t.O t.O 4.0 3.4' t.* S.O 7.0 7.0 S.t 

„ fc.t S.t 2.0 t.O S.O t.C 1.0 7.9 

3.1 7.4 ».0 S.I 1.4 1.0 *■« 



3.9 t.O 0.1 t.t 3.0 1.0 0.1 0.0 l.t t.9 
0.0 Sit 0.0 9.0 O.t t.C 3.3 3.1 3.8 «■« 

3.3 9.0 t.t O.t l.tjt.! 3.0 3.0 9.9 t.t 

t.t t.O o.t t.t t.a s.o i-» o-t o.t t.o 

J.J 0.0 t.t O.t 6.1 1.3 J-' 5 «t 0.0 t.t 

9.9 t.O t.O 9.8 7.9 %C 1.9 9.0 0.9 1.0 

0.1 0.0 0.0 3.0 ?.*,?. - J.a 3.3 t.i o.t 

.5 l.t t .t 8.1 9.1 1." 3.0 t.O 0.1 *tt 

*7.i 1.4 7*t i*o o.lfr.c 3.4 0.3 0.1 

J.t 1.0 7.0 t.O I.l J.: »•* !•< 1 

5.4 t.O S.t t.t S.4U.S '.S 1.3 9.3 0 
7.4 7.0 7.4 3.4 O.ttS.C ».4 t.4 2.t 1 
S.t 3.0 3.9 1.9 1.9'!-* 1 I 7-1 » 

t.l t.a t .i o.t tutlt .t ^ 3 \'* *i* 



I 7.0 7.1 t.l S.t 

3.1 |jUf» J'^ 
4 ?.t 3.4 t.4 t.t 
0 7.1 3.0 t.O 3.0 
?.t t.t t.O t.O 3.0 
S.t 7.0 t.t t.O t.t 
9 t.t t.O t.O 7.0 
9 1.4 t.O t.O 7.4 
t 1,5 t.l 1,5 2.5 
9 7.0 7.4 7.0 
9 7.9 l.t 7.0 
0 l.t 7.t 7.0 
1.9 9.9 O.t t.t 9.0 
1.3 7.9 7.4 t.4 
t.t 1.4 7.0 1.0 
1.0 1.4 1.9 7.0 .4 
t.t 1.1 0.0 0.9 0.0 



9 7.9 J.t t.O 7.4 

3.9 7.0 S.4 t.t 7.4 

7.9 7.9 3.0 t.t 7.4 

9.0 7.0 3.4 S.4 7.4 

7.4 7.0 3.0 S.O 7.4 

".a t.o t.o t.o t.o 



7*4 7.4 9.0 9.6 0.9 
4.0 3*t 9.0 0.0 0 
9 7.4 0.9 9.1 0 



2.9 X.t *«4 S.t 9*9 

- J.t 7.0 S.t •** 

7.4 t.t t.t 0.0 

S.t 7.1 t.t 0.1 

7.1 3.4 3.4 7.4 .4 

t.t l.t 7.4 7.9 .4 

t.4 7.0 3.0 7.4 - * 



•t l.t 0.3 1.11,1 Sti Z:9 MM M,'.? ?tt It9 tit t't 
.0 9,1 1.1 0.9 0.1! t.t 1.9 t.t S.t S.tjS.O 1.4 0.9 0.0 0.0 



*t S.O t.t t.t t.t 

.1 J.O 3.0 J.O .4 

l.t 7.0 S.t S.O t.O 

4 t.O 7.0 ?. t p.O 



1.4 0.3 l.t 0.9 S. 113.0 t.9 . 4 3.4 7.9 

7.9 3.9 t.t t.9 t.l 3.1 t.4 1.4 3.4 1.9 

7.4 0. 3 9.9 9.0 .4^ 0.0 1.9 .4 1.4 t.O 

7*4 0.0 9.0 0.9 9.919.0 t.l 1*4 S.4 t.9 

7.4 t.t t.t t.t t.t, t.t t.l t.t S.t 3.4 

7.4 t.O l.t t.t .4' 3.1 1.4 7.0 3,4 *.t 

j jt a.t t.t fl.a .4 a.t t.t 7.0 S.t *».5, 

4 0.0 0.9 0.8 l.t 1.0 7.4 t.t 7.4 J.t 

t.4 t.O 1.0 0.0 t.O 3.1 S.4 7.4 S.O t.t 

1.4 0.0 1.0 0.0 9.9 0.0 t.9 7.0 S.O t.O 

7.4 9.9 0.9 0.0 9.9 t.O t.9 3.9 7.4 t.9 

7.4 9.0 9.0 0.0 I.l 0.0 t.4 3.4 S.O t.O 

9.9 0.9 0.9 0.0 S.t t.O 7.4 S.4 

7.4 3.9 0.9 0.0 t.O 0.0 t.9 S.9 7,4 t.9 

7.4 J'MA'J-A'lMX 

.0 t.t 1.0 t.t .4 9.0 7.4 7.0 S.O S.V 

.4 9.0 t.t 1.8 t.t t.t 7.9 t.t 7.4 3.1 

t.4 0.9 0.9 0.9 9.0 0.0 3.9 l.t S.O t.O 

9 9.0 3.9 9*9 0.9 0.0 7.0 t.O S.O 3.4 

0 t.O 9.9 t.t 9.3 t. 6 7.t t.O S.t 7.4 

4 t.t t.t 0.9 1.0 9.1 J.t .4 3.0 1.4 

t^O - }*^:! l.t 9.0 



3.0 0.9 3.9 0 

3.4 3.9 9.0 O.fi 0.0 

S.t 3.4 0.0 9.0 0.0 

S.4 3.4 0.0 t.t 9.0 

S.t 3.4 9.6 I.l 0.0 

t.t 3.4 9.0 t.t t.t 
l.t l.t 



»4 3.4 i t : 

3.9 7.4 l.t 



9.C t.a 



J.t 7.1 7.« t.t 
S.t t.4 7.9 t.t 
S.t 7.0 7.4 t.t 
7.t 3.9 3.9 t.O 
S.O S.t 7.4 
S.t 7.t 7.4 
t.O 7.9 3.4 7.0 .t 
t 7.0 S.t 7.0 
4 0 



0.0 0.0 0.9 9 

0.9 0.0 0.9 0.9 9.9 

0.0 t.O 9.0 O.t 9.9 

0.0 0.0 0.0 0.0 t.t 

t.t 0.9 0*0 O.t t.l 

0.0 0.0 0.0 0.0 0.0 
t.O O.t l.t 0.1 0.0 



0.1 o.t t.t t.t t.i 

1.0 0.0 t.t 0.1 I 

t.t t.t l.t 0.0 o.o 

6.0 O.t 1.0 0*0 0.0 

i.o o.o i.a o.o i.i 

t.t O.t l.t 0.0.9.1, 



U «« U W ft ■ ■*« W * ■* * * •* - * - __• 7- -- _ ■ i 

S ' S.t o.f g .l t; « M .T3*T-|~nPKT 3TT 

t.l t.O O.t 0.0 O.Oit.C 3.9 3.1 S.t t.t 

1.9 t.t t.t t.t t.t't.t 3.1 0.3 t.t 0.0 

.4 t.t 9.0 0.0 t.tll- 1 »•« 9.« 

t.9 t.O 7.9 1.0 7.9 7. C t.8 3.0 9.9 S.O 

t.O 0.8 O.t O.O O.O 1 !.* 3.3 0.0 0.0 0.0 

7.0 0.0 9.0 O.O O.ojt.O 3.1 -4 9-0 6.0 

t.9 9,0 0.9 9.9 Ml).! 3.0 9.3 0.0 0.9 

,4 t.O 0.3 t.t t.O't.l 3.9 8-9 9.9 t.3 

9.9 t.O 0.0 9*0 t.9il.f 5.1 3.9 9.3 9.9 

0.9 l.t t.3 0.0 0.8't.O 0.1 4.t t.t o.i 

1.3 O.t 9.9 t.9 O.Sjl.t 3.3 0.0 0.8 O.t 

«.l O.t S.I 0.0 i.t't.O 3.3 t.t 3.0 t 



9,4 0.0 t.t 0.0 

t.l 3.4 O.O O.t 0.0 

3.0 J.t t.O 9.0 0.0 

t.O t.t O.t .4 O.t 

O.t S.O t.t t.t t.l 

4.0 7.0 S.t 3.9 t.t 

t.O 3,4 t.t l.t t.4 

t.i ?.t t.O l.t 0 ^1 

U% S.'t 0.9 .4 l.t 

t.O ?.t 7.0 7.t l.t 

t.t 7.0 O.t t.4 l.t 

t.O 7.1 t.3 l.t t.O 

0.9 t.t t.t 7.4 0.0 

O.t 3.4 t.O t,t «.t 

8.8 I.l I.l I.l I.l 



7,4 t.O t.t *.4 t.4 
t.| S.O S.t t.9 t.4 
t.4 t.O 3.4 t.O 4.1 
?.t t.t t.t t.t S.O 
1.0 t.O t.O 3.4 t.O 
0.0 3. 0 7.0 7.0 1.0 
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